Use of Representative Operation Counts in Computational Testings of Algorithms

In the mathematical programming literature, researchers have conducted a large number of computational studies to assess the empirical behavior of various algorithms and have utilized the CPU time as the primary measure of performance. Generally, CPU times are not a good measure of an algorithm's performance because they are implementation dependent, hard to replicate, and they do not provide much insight about an algorithm's behaviour. In this paper, we illustrate the notion of "representative operation counts" that can complement the conventional CPU time analysis and can allow us (i) to identify the asymptotic bottleneck operations in an algorithm, (ii) to estimate its running time for different problem sizes and on various computers without actually implementing it, and (iii) to obtain a fairer comparison of several algorithms. We believe that our ideas can be easily incorporated in an empirical study and can yield valuable insight about algorithms' behaviour. * This paper appears as a chapter in our forthcoming book, Ahuja, Magnanti, and Orlin [1993].